Data Management in Microservices: State of the Practice, Challenges, and Research Directions

We are recently witnessing an increased adoption of microservice architectures by the industry for achieving scalability by functional decomposition, fault-tolerance by deployment of small and independent services, and polyglot persistence by the adoption of different database technologies specific to the needs of each service. Despite the accelerating industrial adoption and the extensive research on microservices, there is a lack of thorough investigation on the state of the practice and the major challenges faced by practitioners with regard to data management. To bridge this gap, this paper presents a detailed investigation of data management in microservices. Our exploratory study is based on the following methodology: we conducted a systematic literature review of articles reporting the adoption of microservices in industry, where more than 300 articles were filtered down to 11 representative studies; we analyzed a set of 9 popular open-source microservice-based applications, selected out of more than 20 open-source projects; furthermore, to strengthen our evidence, we conducted an online survey that we then used to cross-validate the findings of the previous steps with the perceptions and experiences of over 120 practitioners and researchers. Through this process, we were able to categorize the state of practice and reveal several principled challenges that cannot be solved by software engineering practices, but rather need system-level support to alleviate the burden of practitioners. Based on the observations we also identified a series of research directions to achieve this goal. Fundamentally, novel database systems and data management tools that support isolation for microservices, which include fault isolation, performance isolation, data ownership, and independent schema evolution across microservices must be built to address the needs of this growing architectural style

Fast Search-By-Classification for Large-Scale Databases Using Index-Aware Decision Trees and Random Forests

The vast amounts of data collected in various domains pose great challenges to modern data exploration and analysis. To find "interesting" objects in large databases, users typically define a query using positive and negative example objects and train a classification model to identify the objects of interest in the entire data catalog. However, this approach requires a scan of all the data to apply the classification model to each instance in the data catalog, making this method prohibitively expensive to be employed in large-scale databases serving many users and queries interactively. In this work, we propose a novel framework for such search-by-classification scenarios that allows users to interactively search for target objects by specifying queries through a small set of positive and negative examples. Unlike previous approaches, our framework can rapidly answer such queries at low cost without scanning the entire database. Our framework is based on an index-aware construction scheme for decision trees and random forests that transforms the inference phase of these classification models into a set of range queries, which in turn can be efficiently executed by leveraging multidimensional indexing structures. Our experiments show that queries over large data catalogs with hundreds of millions of objects can be processed in a few seconds using a single server, compared to hours needed by classical scanning-based approaches

‘Mutirão Agroflorestal’: herramienta de red de agroforestería del Vale do Paraíba, Brasil

Difundir los sistemas agroforestales (SAFs) en la cuenca del río Paraíba do Sul, Estado de São Paulo, Brasil, es el propósito de esfuerzo conjunto en Polo Regional/APTA, en Pindamonhangaba. Los SAFs se basan en consorcios de cultivos agrícolas, arbustos y árboles, aprovechando los mismos recursos (agua, luz y nutrientes). Entre los años 2010-2013, unas 330 personas de diversos orígenes participaron en esfuerzos conjuntos agroforestales. La metodología participativa incluye la planificación para la preparación y manejo de los SAFs, después de conciencia a través de conferencias y excursión. El método "mutirão agroforestal” promueve el intercambio de conocimiento, el rescate de los saberes populares y los recursos genéticos. El enfoque de SAFs en la restauración de las áreas ribereñas con producción diversificada amplificando la biodiversidad de la selva atlántica. La producción se basa en especies comestibles, entre ellos: Maranta arundinaceae, Colocasia esculenta, Pereskia sp., Talinum paniculatum, Xanthosoma sagittifolium, Manihot esculenta, plátano resistente - Musa sp.; de árboles nativo para madera (Calophyllum brasiliense e Centrolobium tomentosum), frutas (Euterpe edulis, Rolinia mucosa) y plantas fertilizantes (Cajanus cajan, Flemingia macrophylla, Tithonia diversifolia, Gliricidia sepium, Sesbania sp., Inga sp.). Se evalúan indicadores de sostenibilidad para balisar la gestión: resistencia a la penetración de una barra de hierro, tasa de cobertura del suelo por la proyección de la plantas de dosel, cantidad y calidad de vegetación espontánea, la presencia de organismos vivos en la capa superior del suelo, contenido de materia orgánica por la reacción del suelo con peróxido de hidrógeno, calidad de especies de anclaje y la cantidad de hojarasca del suelo. Como resultado, un grupo multidisciplinario ha sido articular la formación de Red de Agroforestería del Vale do Paraíba, de difundir los SAFs en la cuenca y los recursos genéticos de estas unidades pasan a los productores

Pervasive gaps in Amazonian ecological research

Biodiversity loss is one of the main challenges of our time,1,2 and attempts to address it require a clear un derstanding of how ecological communities respond to environmental change across time and space.3,4 While the increasing availability of global databases on ecological communities has advanced our knowledge of biodiversity sensitivity to environmental changes,5–7 vast areas of the tropics remain understudied.8–11 In the American tropics, Amazonia stands out as the world’s most diverse rainforest and the primary source of Neotropical biodiversity,12 but it remains among the least known forests in America and is often underrepre sented in biodiversity databases.13–15 To worsen this situation, human-induced modifications16,17 may elim inate pieces of the Amazon’s biodiversity puzzle before we can use them to understand how ecological com munities are responding. To increase generalization and applicability of biodiversity knowledge,18,19 it is thus crucial to reduce biases in ecological research, particularly in regions projected to face the most pronounced environmental changes. We integrate ecological community metadata of 7,694 sampling sites for multiple or ganism groups in a machine learning model framework to map the research probability across the Brazilian Amazonia, while identifying the region’s vulnerability to environmental change. 15%–18% of the most ne glected areas in ecological research are expected to experience severe climate or land use changes by 2050. This means that unless we take immediate action, we will not be able to establish their current status, much less monitor how it is changing and what is being lostinfo:eu-repo/semantics/publishedVersio

Pervasive gaps in Amazonian ecological research

Biodiversity loss is one of the main challenges of our time,1,2 and attempts to address it require a clear understanding of how ecological communities respond to environmental change across time and space.3,4 While the increasing availability of global databases on ecological communities has advanced our knowledge of biodiversity sensitivity to environmental changes,5,6,7 vast areas of the tropics remain understudied.8,9,10,11 In the American tropics, Amazonia stands out as the world's most diverse rainforest and the primary source of Neotropical biodiversity,12 but it remains among the least known forests in America and is often underrepresented in biodiversity databases.13,14,15 To worsen this situation, human-induced modifications16,17 may eliminate pieces of the Amazon's biodiversity puzzle before we can use them to understand how ecological communities are responding. To increase generalization and applicability of biodiversity knowledge,18,19 it is thus crucial to reduce biases in ecological research, particularly in regions projected to face the most pronounced environmental changes. We integrate ecological community metadata of 7,694 sampling sites for multiple organism groups in a machine learning model framework to map the research probability across the Brazilian Amazonia, while identifying the region's vulnerability to environmental change. 15%–18% of the most neglected areas in ecological research are expected to experience severe climate or land use changes by 2050. This means that unless we take immediate action, we will not be able to establish their current status, much less monitor how it is changing and what is being lost

[pt] CRIAÇÃO AUTÔNOMA DE ÍNDICES EM BANCOS DE DADOS

A escolha e materialização de índices são atividades comumente realizadas por administradores de bancos de dados (DBAs) para acelerar o processamento de aplicações de bancos de dados. Devido à complexidade da tarefa de seleção de índices e à pressão por maior produtividade sobre os profissionais que realizam sintonia, diversos trabalhos na literatura e em sistemas comerciais procuram obter ferramentas que possam apoiar o DBA na escolha dos melhores índices para uma dada carga de trabalho. Classificamos estes trabalhos como sendo de auto-sintonia local, uma vez que se focam em um problema de sintonia específico, em oposição a trabalhos de auto-sintonia global, que almejam obter um desempenho aceitável para o sistema como um todo. Esta dissertação propõe duas arquiteturas que permitem automatizar completamente a sintonia de índices. A indepedência de intervenção humana é obtida através do uso de agentes de software. A combinação de agentes com SGBDs torna os sistemas mais autônomos e capazes de auto-sintonia. Implementamos uma das arquiteturas propostas no SGBD de código fonte aberto PostgreSQL e obtivemos resultados experimentais com uma carga transacional que mostram a viabilidade de nossa abordagem.The choice and materialization of indexes are activities commonly done by database administrators to speed up database application processing. Due to the complexity of the index selection task and to the pressure for productivity increase put on tuning professionals, many works on the literature and on commercial systems seek for tools that can help the DBA choose the best indexes for a given workload. We classify these works as local self- tuning, once they are interested in a specific tuning problem, in opposition to global self-tuning work, which is targeted at obtaining acceptable performance for the system as a whole. This dissertation proposes two architectures that allow the complete automation of the index tuning task. Human intervention independence is achieved through the use of software agents. The combination of agents and DBMS makes systems more autonomous and self-tuning. We have implemented one of the proposed architectures in the open source DBMS PostgreSQL and obtained experimental results with a transactional workload that show the feasibility of our approach